Their key insight is that supervised fine-tuning with a set of surprisingly small but extremely visually appealing images can significantly improve the

generation quality. Effective fine-tuning of LLMs can be achieved with a relatively small but high-quality fine-tuning dataset, e.g., using 27K

prompts.

They increase the channel of AE from 4 to higher dimension. They use additional adversarial loss for reconstruction and also they apply a non

learnable preprocessing step to RGB images using a fourier feature transform to lift the input channel dimension.

They use a large U-Net with 2.8B trainable parameters. They increase the channel size and number of stacked residual blocks in each stage for

They pre-train the model with 1.1B images. They train the model with progressively increasing resolutions. This approach improve finer details at

They introduce a dual approach involving model miniaturization and a reduction in sampling steps. The methodology leverages knowledge distillation

to streamline the U-Net and image decoder architectures, and introduces an innovative one-step DM training technique that utilizes feature

matching and score distillation.

VAE Decoder Optimization: Utilizing a pretrained diffusion model F to sample latent codes z and a pretrained VAE decoder to reconstruct images x,

we introduce a VAE Distillation (VD) loss for training a tiny image decoder G. they only build the G with CNN blocks to eliminate complex

components like attention and normalization (I do not know why they think that norm layers are computationally overwhelming.)

reproduce the original model’s intermediate feature maps and outputs effectively. Initializing noises and sampling images with an ODE to get noise

image pairs results in low quality images. So they use Rectified Flow tackles this challenge by straightening the sampling trajectories. Using MSE

narrow the range of feasible outputs using existing finetuning methods like LCM.

One Step Training: They first trained model for feature matching with SSIM Loss as warm-up. While at this stage, they sample noise - image pairs.

As trajectory of ODE’s (For example DDPM) are not straight, they used LCM-Lora for rectifying the flow. They say that warm-up training results are

To achieve better quality at low step size, they propose to distill along the student’s backward path instead of the forward path. Put differently,

rather than having the student mimic the teacher, They use the teacher to improve the student based on its current state of knowledge. we propose

a Shifted Reconstruction Loss that dynamically adapts the knowledge transfer from the teacher model. Specifically, the loss is designed to distill

at lower time steps. They propose noise correction, a training free inference time modification that enhances sample quality.

Commonly chosen s.t. xT is not pure noise during training, but rather contains low-frequency information leaked from x0. xt = αtx0 + σtxT, here

the leakage reason, any stochastic interpolant xt, t < T still contains information from the ground-truth sample via the first summand αtx0.

by simulating the inference process during training, which can also be interpreted as calibrating the student on its own upstream backward path.

They first perform backward iterations of the student model to obtain , then use this as input for

image features enhanced fine-grained details while maintaining the same overall structure as the student’s prediction.

Sampling from DMs can be seen as solving a differential equation through a discretized set of noise levels known as the sampling schedule. They

propose a general and principled approach to optimizing the sampling schedules of DMs for high-quality outputs.

Authors says that Adversarial Diffusion Distillation was a big move but usage of the fixed and pretrained DINOv2 network restricts the discriminator’s

Sampling from DMs can be seen as solving a differential equation through a discretized set of noise levels known as the sampling schedule. They

propose a general and principled approach to optimizing the sampling schedules of DMs for high-quality outputs.

Sampling from DMs can be seen as solving a differential equation through a discretized set of noise levels known as the sampling schedule. They

propose a general and principled approach to optimizing the sampling schedules of DMs for high-quality outputs.

Sampling from DMs can be seen as solving a differential equation through a discretized set of noise levels known as the sampling schedule. They

propose a general and principled approach to optimizing the sampling schedules of DMs for high-quality outputs.